All You Need is the Monad. . . What Monad Was That Again?

Probability enjoys a monadic structure (Lawvere 1962; Giry 1981; Ramsey and Pfeffer 2002). A monadic computation represents a probability distribution, and the unit operation return a creates the (Dirac) distribution “certainly a.” The bind operation combines a distribution of type M a and a function of type a -> M b; the function is a probability kernel (Pollard 2002), and it represents the conditional probability of b given a. The bind operation produces a new distribution which is defined using a Lebesgue integral, also called an “abstract” integral, over all possible values of type a. The Lebesgue integral works out in the same way whether the measurable space of a’s is discrete, continuous, or hybrid. To write interesting distributions, one must include in one’s monad one or more probabilistic operations, like discrete choice or a primitive distribution. Useful primitive distributions include a biased coin (Borgström et al. 2013), the uniform distribution over the unit interval (Park, Pfenning, and Thrun 2008), normal, beta, gamma, and other distributions. Probability monads inform the design of several programming languages, including some cited above. But a probabilistic programming language must do more than just represent probability distributions; it must support inference. In Bayesian inference, a term in the language denotes a prior distribution, some information is observed, and using the prior distribution and the evidence provided by the observation, we calculate or estimate a posterior distribution. This operation is sometimes called conditioning. Inference and conditioning raise design questions that appear not to have canonical answers. Here are some: